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ABSTRACT 

Tl^e correlation '•between high- inference measures and 
low-inference observation measures was studied using two groups Of 
social studies student teachers (25 students per group) . The 
high- inference measure for tWo video-taped lessons was obtained with 
the 10-item Teacher Performance Appraisal Scale (TPAS) , on which 
pupils rated the lesson on a scale from one to seven. The 
lowrinf erence observation measures were obtained by having trained' 
observers categorize the videotapes into the 10 categories of 
Classroom Interaction Analysis (IA) observation system of Ned A. 
Flanders. Separate analyses were made for each group. No steady'* 
correlations were established between the high-inference and 
low-inference data. Another study was then conducted in whibh scores 
for the individual items comprising the TPAS were developed^ and. the 
results were subjected to additional correlation stu dies with the I A 
scores. The only consistent correlation was a negative correlation 
across observations for the high- inference variable "content 
..organ izedH. and the low-inference variable "student talk — initiation." 
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RELATIONSHIP OF H^GH -INFERENCE AND LOW-INFERENCE j 

; • ' ‘ . . ' ' . ■ i 

. OBSERVATION MEASURES - i 

♦ * 1 • i 

, '*•, „ . .. 1 * l 

■ t v . ’ > t 

A search to establish correlations between high-inference data, such as 

those obtained through fairly specific questionnaires] and lower- inference" 

* 4 * . _ . , 

'data, such as those obtained through the use of observational category systems, 
is useful for several reasons. Many of the variables discussed in the literature 

i ' • f • 

. that have consistently related to student achievement were secured by the use of 

. V V • 1 

high-inference gating scales, e.g. , clarity, variability, enthusiasm,' etc. 
However, the lists of specific teacher behaviors describing these variables have 

not been developed. If, within those high-inference variables found to cor- 

' <> * % • ’ \ / 

■ 1 9 c * f t 

relate most consistently with student achievement, certa'in specific Jteacfr?fr~ 
behaviors can be isolated, the task of teaching these skills to pre-service and 



in-service teachars becomes considerably, easier. At present many ''colleges and 






i^r 



universities do teach their beginning teachers- specific teacher behaviors, i.e., 

- - f 

questioning skills., reinforcement techniques, etc. The justification for the 

\ ' * l- # 

selection of these behaviors appears at present to be based main ly_on expert 
belief and seldom on experimental or correlational- studies linking the specific 

Or . * 

behavior to student achievement." The rising trend to vise "performance criteria" 
in the training of teachers indicates that more schools will orient their 

* ^ ’ i , / ’ . 7 

.programs toward specific" teacher behaviors; A knowledge of which teacher 

\ 

behaviors relate to demonstrated effective teacher characteristics should 
* \ 
enable curriculum developers to select specific behaviors for inclusion in 

their programs with mor6 precision and confidence. , 

The data for this study were obtained from a previous study of two groups 

' 2 ’ 

of social studies student teachers. Each group contained twenty-five students 

^ V 

and was observed twice, during the first week and during the sixth or/last week 
j • • * 
of student teaching. The design of the study called for a thirty-minute video- 

/ > * * 

tape to be made of each observed lesson followed by a pupil evaluation of that 

. \ ; • 



V 



lesson. The high-inference measure fot each lesson was obtained with the ten- . 
item Teacher Performance Appraisal Scale' (TPAS) developed by Horace E. Aubertine 

t ' - . 

and William D. Johnson. These high-inference. measures were determined by asking <i 

f 1 V * * ' 

* , L % % ' ‘ Q 

the pupils to rate. the lesson using a scale from one to seven. The variables 

' ■ ' 

upon which these measures were obtained can be summarized as follows: 

. • ; ' \ 

1. Learning aims understood ^ - 6. Method stimulating 



•2. Learning aims developed 7. Method successful 

3. Content meaningful t 8. . Contact with learner • ' 

N 4. Content organized • 9.‘ Participation encouraged 

5. Mfethod used appropriate 10. Feeling of accomplishment t 

The last ten minutes of each period, observed were reserved for use by the 

investigator who. read a definition of each of the high-inference^ measures £nd 
«S> ... . • ' 

attempted to describe the results a pupil should have seen for various -levels of 

% ► . % f o * n 

each measure. %hese uniform definitions of the individual items attempted to 

• • * * 

• standardize the information given the pupils by the several investigators involved 

.. •• ■ ■ ■ • 

in the study. . *■ 

The low-inference observation measures were obtained by having trained 



observers categorize the videotapes into * the ten categories of the classroom 

interaction analysis (IA) observation system developed by Ned A. Flanders.^ 

■» / 

The observers were trained using the instructions and. ground rules contained in 
the manual developed by Edmund J. Amidon and Ned A. Flanders.^ The ten cate- 



gories used in. this system dqn be summarized as follows: 



1. 


Accepts feelings 

\' • 

Praises or encourages 


.6. 


Giving directions 


1 2 - 


7. 


Criticizing or justifying authority 


■ . 3. 


Accepts or uses student ideas 


8. 


Student talk--respqnse 

* C * 




Asks questions 


9. 


0 

Student talk--initiation 


: 5. 


Lecturing • 


10. 


t 

Silence or confusion 



3 






In this 'system the observer- determines at 1 three.-second intervals which category 
is being usedv The resulting tallies were processed by computer program t;o • J 

* c r:l ’ i . 1 ’ 

- . ; SV , • •• * o 

develop totals, percentages, and selected ratios for each -lesson. 

Ned Flanders, the developer of this low-infe.rence observation system, feels 
that the use. of these measures can.help'to explain Why- the results. wei^e obtained . 

c. „ * 

in studies on teaching effectiveness and, in fact, lists as one of .the two 

* * 

questions his latest book on teaching' behavior attempts to speak to as, "Can 

r / • /" . • , . 

objective data provide insights into subjective problems?" His discussion of 

this concept concludes with this statement, "It may take years of research and „ • 

\ • - 

development before we can synthesize the subjective and objective elements, 
but there can be no escape from confronting the question."^ We have attempted 
to confront the question; and our results have been confusing, disappointing, 
and suggestive for future studies. . - • • 

Because the two groups of social studies studerit teachers differed in the 
training they had prior to, student teaching, separate analyses were made for 
each group. We hoped to find significant relationships between the high- 

~ r 4 1 . m # , . I 

# 1 % i 

inference and IA variables which would be consistent across the two groups and 

( r- . * * . . r ♦ * , 

. ' ' / 

across the two occasions. Table I indicates the results, showing any significant 
'correlations between one of the Flanders IA categories and the high-inference 
variables. Thus, looking at the- table, you can assume that there were , no sig- 
nificant correlations“at all between Flanders’ IA categories 1, 4, 6, 7, and 9. 

, * ' ’ • . / 

The .TPAS .instrument has been factor arial y aed. tf y^i 1 1 iam D. Johnson, University 

of Illinois, who developed a TPAS-A <: Heognit;ive oriented factor and a TPAS-B 

• • . « ' • ' ) 

affective oriented factor from the TPAS variables. These two factors were also 

" O 

- 4 

used in this- analysis. ' • • 

* • » \ 

The original study secured observations on another- high-inference measure, 

\ 1 ' !*.••■» ' * 

0 V 

the Illinois Teacher Evaluation .QuestiohnaU-re (ITEQ) , developed by the late 

‘ - • / V- . : *• .. • 

Richard E. Spencer, University- of Illinois. This instrument was administered 

- . ' ' ■ .. - ■ ' . . 



• .1 



p 



/ 



‘ b 



/ • 



' i > 




onlyonce at- the completion of student teaching. The ITEQ secured a/meaSure of • 

• ' " ' /• v 

pupil evaluation ‘of total ‘teacher performance,' e.g., tests, homework, textbooks,, 
etc. The significant correlations 'for this measure and^th£ k ' four ITEQ sub-facto.rs 
created by Spencer are also included in Table I. ./ 

Our search to establish correlations be tween/the '•high- inference measures of 

/ 

the TPAS and. ITEQ and the /lower-inference measures of Flanders IA received little - 
support from this initial analysis. An examination of Table I reveals that there 

. . v 

i\ no correlation between a Flanders behavior and a TPAS which holds' up across 
Lrials and across groups. The cprrelations that do exist would/appear to have 
imited generality. * 

. A'' • ‘ . . '' 

In an attempt to include only those student teachers for whom we had 

complete measure on all three scales, the number of teachers used was reduced to 

a total of seventeen from both groups. # As the ITEQ measure of .total teacher 

> ^ '* 

performance again gave only scattered and inconclusive correlations and was 

* . . . 

administered only at the conclusion of the student teaching experience, our 
attention was focusedupon the two measures available for both observations-- 

* ^ 9 

Flanders and TPAS. Table II indicates the results of this analysis. And, again, 
we w^re unable to get, for this kind of population, steady correlations between 
the Flanders variables and high-dnference measures. Correlations were developed 
for several IA categories for the first observation, but none of these held up 
across observations. - v A comparison - of the mean scores for each of the TPAS and 
IA variables across the two occasions explains some of the problem.. Tables III 
and IV show the means and standard deviations of the twenty variables across 
trials. ' 

Observation indicates that teacher scores on the' high-inference measure ' 






K 



were slightly .lower on the second administration. Ta the extent that pupil 

• • ■ / . r. 

evaluation of these lessons is also a measure of attitude toward the student 

/ • ( - . 
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teacher, this decrease in positive attitude is a fairly universal phenomenon 

s ■ . ' ' 

and might be expected in any longitudinal study. In contrast, the variables 

on the Flanders category system moved both up and down; indeed, when ah entire 

* 

category system is considered, the scale is ipsative. Thus, as we now realize, 

a * * 

... i 

the scores on the student rating form (JPAS) were, moving down across time, 
whereas, the scores bn" the category system were moving in an ipsa'tive manner. 

Given ’these phenomena, one might, expect That in other studies in which both 

* / 

f * . - / 

ratings and specific counts were obtained on the same teachers at two points 
in time , the correlations would not be consistent across occasions. 

An examination of 'Table IV shows that consideration might' be given to the 

. 1 .. ■C* . ■ 

use of other measures of cc .tral tendency and deviations for- data containing 

categories with such extreme measurements. The median could be expected to 
yield a more representative value;, particularly in the case, of categories six 
and nine which indicate sigmas greater than the mean for both, observations-. 

, * * * , a # 

0 ' ./ .4 ' ^ 

The use of a quartile deviatipn could be combined/ with the median to provide 

l • ‘ r ' . • V • v 

more representative values. • , ' V • 

. % o i. * ^ 

Table V was developed to determine the correlation for each of the IA " 

categories across the two observations. The stability across observations of 

the three main teacher categories used by the student teachers seems to be 

\ ‘ * . v . . : 
revealed by this table. ... • * 

‘ Unable to establish steady .correlations between our high -inference and 

low- inference data, we developed scores for the individual items comprising the . 

high -inference measure (TPAS) and subjected r the results to further correlational- 

studies with the IA low-inference scores. The entire population of student 

t * ' • t 

teachers, available for both obse.rvations , forty for .ithe first and thirty-two for 

t . $ * 

the second, was subjected to a missing-data intercorrelation analysis. 'The 

i . n • ", . , ' 

results of this analysis can be found in Tables VI and VII. We again were unable 



I 






.. * 



.to secure consistent correlations between the variables with but one exception. 

This time we were able to 'achieve a negative, correlation across observations for 

the high-inference variable "Content organized" and the low-inference variable 

; • / / 

"Student talk--initi,ation. " When the- high-inference measure was examined as a 

w * . ' ( 

‘ / - * 

single entity or as one of two factors, this phenomenon remained obscured. It v» 

would seem that, at least for this population of junior and senior high school 

pupils, \^hen .the student teacher conducts a lesson in which the pupils are free 

" N f ' ' ■ 

to initiate verbal behavior, these same students will subsequently judge that 
lesson low on the^ organization of its content. The material read to the pupils 

as they were asked to evaluate the variable "Content organized" of the lesson 

i o', 

stated, "If the teacher makes little or no effort to differentiate between rele- 
vant or irrelevant topics, the lesson is not organized. If you cannot logically 
order the major points of the lesson into a pattern meaningful to. you, the lesson 

b ' i . 

was not organized."*’ In other words, if the pattern or organization of the 

\ * ' * * . ’ - j. , 

lesson was not clear to, the pupil, the lesson deserved a low rating. r £hk cog- 
nitive clarity or-perceived organization of a teacher's’ lesson is one of the few 

C > . 

high-inference variables found to correlate with regression-adjusted student 
achievement scores. ^ The search, for low-inference behaviors which comprise 
"organization" would seem to have revealed a specific classroom behavior, the 
quantity of self-init-iated-student talk, which produces a negative effect on the 

i ' ’ 

perceived level of this Important high-inference variable. Educators, attempting 

to train -their students to present lessons most likely to achieve high ratings 

• . ’ 

on clarity or organization and using methods designed to elicit or encourage selfr 
initiated student talk, should be aware of this finding. ' , 

In conclusion, the difficulties which we encountered in attempting to relate 

r ’ * \ 

high-inference variables to low-inference variables when data are collected at 

}■ ■ ■ * - . 

two points in time may cast doubt upon the results obtained when data are collected 



ERIC 



only once and. may suggest the fallibility of attempts to relate ratings of 



$ 



teachers by students to counts of 



specif 



ic behavior. 



^Barak Rosenshine and N. Furst, "Research in Teacher Performance Criteria," 
in Research in Teacher Education , ed. by B. Othanel Smith (New Jersey: Prentice- 

Hall, Inc., 1971). - « j . 

^Philip C. Limbacher, "A Study of the Effects of Microteaching Experiences 
Upon Practice Teaching Classroom Behavior" (unpublished doctoral dissertation. 
University of Illinois, 1968). «• 

. *■ 

%ed A. Flanders Teacher Influence, Pupil Attitudes and Achievement , Final 
Report Cooperative Research Project No. 397, United States Office of Education 
(Minneapolis: University of .Minnesota, 1960). 

^Edmund Amidon and Ned A. Flanders, The Role of the Teacher in the Classroom 
(Minneapolis;. Paul S. Amidon and Associates, 1963). ^ 

5 F lenders , Analyzing Teaching Behavior (Massachusetts: Addison Wesley 

Publishing Company, 1970). 

^William D. Johnson, "A Discussion of Individual Items of the Teacher 

Performance Appraisal Scale" (University of Illinois, 1967, Mimeographed). 

° o • ' 

^Rosenshine and Furst, "Research in Teacher Performance Criteria," pp. 44- 
45. 
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SIGNIFICANT CORRELATIONS BETWEEN FLANDERS IA CATEGORIES AND 
HIGH -INFERENCE MEASURES (TPAS and ITEQ) 



V 



TABLE XI 



SIGNIFICANT CORRELATIONSBETWREN FLANDERS IA 
AND TPAS FOR ^ELECTED SUBJECTS „ (N * 17) 









First Observation 



Second Observation 



Flanders/ TPAS-Total 

/ 

/ 


TPAS -A 


TPAS-B 


TPAS-Total 


n &S-A 


TPAS-B 


© 

© 


.00 


’ - 00 


.00 


.00 


.00 


2 .49* 


. .53* 


■ ; . tt * 


.20 


.19 ^ 


.19 


.3 .30 


.34 


.22 ' 


.26 


t 

.30 


.15 


4 _.67** 


.73** 


.54 


.03 V 


.04 


v .oo 


5 -.68** 


-.69** 


-.63**^ 


V -.12 


00 


-.35 




, W W 


6 -.08 

< 


-.12 


-.01 


-.08 


-.21 


.17 


7 , .16 


.15 


18 ' 


. 46 • 


.43 


. .45 


8 i .56* 


.59* 


’ .47 


.09 , 

t + 


.09 


.08 


9 .02 


-.03 

l. 

-.21 


* .13 


A 1 

.14 


.02 . 


.36 


10 -.21 

\ 

** p < .01 


' -.20 /.* 


.-.22 

* 


• .-.33 


.05 


* p <. .05 




' . 




.. ■ 


' \ 


! 




TABLE III 


■0 

■ /* * 






- MEANS AND SIGMAS OF HIGH-INFERENCE VARIABLES ACROSS OBSERVATIONS (TPAS) 




First Observation (N “ 40) 


Second Observation 


High -In fere nee Variable 




Mean 


Sigma 


Mean 


Si gnu 


Learning aims understood 


4 


4.93 


.41 


1 ^5^04 


.46 


Learning aims .developed 


♦ 


4.87 


.46 


4. 85 


.49 


Content meaningful 




4.99 


.59 


5f 10 


.59 


Content organized 

\ 




5.27 


.47 


> 4.98 


.56 


Method appropriate 




5.16 


. 52 


5.15 

• V ' 

4.68 


.52 


Method stimulating 

\ 




4.58 


.89 


• 76 


\ \ 

Method successful 

* v t • 




5.14 ■■ 

. 1 ’ 


.49, -J " 


5.02 


.54 



32) 






i 



Contact with learner 
Encourages learner participation 
Feeling of accomplishment 




5.32 



.62 



A 
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^ TABLE IV ' • 

MEANS AND SIGMAS OF LOW-INFERENCE VARIABLES ACROSS OBSERVATIONS (IA) 



First Observation 



; Second Observation 



l 





. 


(N -using 






(N-using 


Low-Inference Variable 


Mean (7.) Sigma Category) 


Mean(7>) 


Sigma 


Category) 


Accepts feelings . 


* .00 


.00 0 

\ ) 


.00 . 


4.00 


• ' 0 


Praises or encourages 


2.47 1 


.81 28 


1.32 


1.10 


21 


Accepts student .ideas 


'3.74 3 


.13 31 


3.60 


* 3.16 


25 


Asks questions 


14.30 6.55 32 

. t 


12.96 


. 6.63 

♦ i 


. 26 


Lecturing 


34.19 20 


.10 \J.32 


33.96 


22.96 


26 


Giving directions 


3.7 ' 5 


.53 23 


2.87 


4.63 


16 ___ 


Criticizing 


' 1.4 1 


.31 13 


.73 


.70 


13 


Student talk--respons.e 


21.24 14.20 32 


■ , 24.50 

j 


• 15.91 


26 , 


Student talk- -initiation 

. • • 


14.9 19 


.10 , 29 


12.71 


15.76 


23 ^ 


^ \ * 

Silence or confusion 

’ ' • ' 3 ■ . 


7.76 5.50 , 32 ' 


10.70 


8.12 


26 


c 

% ■ 1 


i 


* i 

rABLE V 








CORRELATIONS 


OF LOW -INFERENCE VARIABLES ACROSS OBSERVATIONS 


* } 


Low-Inference Variable 


^ Correlation Coeff 


• 


N-us 


ing Category 


"‘■.‘1 / 


‘ (Trial I with Trial II) 


•- 


- ‘ , 


. 1 . y 

Accepts feelings 


* 


4 .00 

\ 






0 


Praises or encourages 


» 


.34 


- 
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